The technology described herein relates to data processing systems and in particular to display controllers for data processing systems.
FIG. 1 shows an exemplary data processing system that comprises a central processing unit (CPU) 7, a graphics processing unit (GPU) 2, a video codec 1, a display controller 30, and a memory controller 8. As shown in FIG. 1, these units communicate via an interconnect 9 and have access to off-chip memory 3.
In use of this system, the GPU 2, video codec 1 and/or CPU 7 will generate surfaces (images) to be displayed and store them, via the memory controller 8, in respective frame buffers in the off-chip memory 3. The display controller will then read those surfaces as input layers from the frame buffers in the off-chip memory 3 via the memory controller 8, process the input surfaces appropriately and send them to a display 4 for display.
FIG. 2 shows an exemplary data path for the input surfaces for display in the display controller 30. It is assumed in this example that the display controller 30 can take as inputs for a given output surface to be displayed, a plurality of input surfaces (layers), and includes, inter alia, a composition engine (stage) 22 that is able to compose one or more input surfaces (layers) e.g. generated by the GPU 2 and/or video codec 1) to provide a composited output frame for display.
As shown in FIG. 2, the display controller 30 includes a DMA (Direct Memory Access) read unit 20 that reads data of input surfaces to be displayed and provides that data appropriately to respective layer processing pipelines 21 that perform appropriate operations on the received input surfaces before they are provided to the display composition stage 22, where they are composited into the desired composited output surface for display.
The composited output surface (i.e. the frame that is to be displayed) is then subjected to display timing control 23 (e.g. the inclusion of appropriate horizontal and vertical blanking periods), and then provided to the display output interface of the display controller 30 for provision to the display 4 for display.
This process is performed for each frame that needs to be displayed, e.g. at a rate of 30 or 60 frames per second.
As such display processing is a real-time operation, the display controller 30 needs to deliver the pixel data to be displayed to the display 4 (to the display output) regularly, in accordance with the desired display refresh rate.
The Applicants have previously proposed in their UK Patent Application GB-A-2524359 an improved display controller that includes functionality to allow composited frames output by the composition stage 22 to be written back to memory, as well as being provided to a display for display.
FIG. 3 shows an exemplary data path for the composited output surface when being written back to memory by the display controller in display controller arrangements of the type proposed by the Applicants in their earlier UK patent application.
As shown in FIG. 3, the write-out unit 31 of the display controller includes a local buffer 40 and DMA (Direct Memory Access) logic 41. The composited output surface data from the display composition stage 22 is streamed (fed directly) to and buffered in the local buffer 40 of the write-out unit 31 as that data is being generated, and then written from the local buffer 40 to a frame buffer 33 in, e.g., external, memory, by the DMA write logic 41.
The local buffer 40 (which may, e.g., be in the form of a FIFO) provides “latency” buffering (latency “hiding”) in the memory write-back path to allow for potential latency in writing the composited output surface data to the memory. This is desirable to allow, for example, for the fact that, as discussed above, the composited output surface will be being generated by the display composition stage 22 at a rate that is governed by the requirements of the display, whereas the rate at which the DMA write logic 41 of the write-out unit 31 will be able to write composited output surface data to the frame buffer 33 in memory will be governed by the rate at which that data can be written to the memory. There may therefore be a mismatch between the rate at which the data can be written out by the write-out unit 31 and the rate at which that data is provided to the write-out unit 31. The local, “latency-hiding”, buffer 40, is present in order to allow for that.
The provision of functionality to allow composited frames output by the composition stage 22 of a display controller to be written back to memory facilitates a number of improved display controller operations.
For example, where the display controller is compositing plural input surfaces to provide an output surface for display, then the composited output surface can be written back to memory, and if the input surfaces have not changed (which can be a relatively common occurrence for certain content), then for the next frame to be displayed, the written-back composited output surface can simply be read in and displayed again, rather than having to read in and recompose the plural separate input surfaces, thereby providing savings in bandwidth and power when displaying the next frame or frames.
FIGS. 4 and 5 illustrate this operation and show a display controller 30 that additionally includes, as shown in those figures, a DMA write-out unit 31 that is operable to be able to write the output from the display composition stage 22 to a frame buffer 33 in the external memory 3.
FIG. 4 illustrates the operation where three input surfaces 34, 35 and 36 are read by the display controller 30 and processed to provide a composited output surface 37 which is both provided for display output and written by the DMA write-out unit 31 back to a frame buffer 33 in main memory 3.
FIG. 5 then shows the operation for the next frame, in the case where it is determined that the frame to be displayed has not changed (e.g. by identifying that the input surfaces and their desired composition has not changed). In this case, as shown in FIG. 5, the display controller 30 simply reads the already composited output surface 37 from the main memory 3 and provides that directly for display. In this case, as only one input surface is being read, there is a reduction in the memory bandwidth and processing required for that operation.
Also, as shown in FIG. 5, only one of the layer processing pipelines 21 needs to be active for processing the single, already composited, input surface (i.e. the already composited output frame 37 can be processed as a single layer by the display controller 30). This again provides further power and processing savings.
As well as using memory write-back of a composited frame to reduce the amount of processing that may be required when displaying a sequence of composited frames as shown in FIGS. 4 and 5, the memory write-back operation of the display controller 30 can also be used for other purposes.
For example, the memory write-back operation can be useful in the situation where plural displays are provided and used to display the same output surface, e.g. where an output surface to be displayed on the data processing system's local display is also to be displayed on (“cloned” to) a second, e.g. external, display. It is often the case in such arrangements that the external display will require and use a different resolution and/or aspect ratio to the local display, and so, typically, two separate output surfaces for display would be generated and processed by respective display controllers and/or display processing cores, one for each display.
However, in arrangements where the display controller has memory write-back capability, then a first display controller may be operated to generate the image for display, and output it, e.g., to the local display, together with writing the generated output surface to memory. The output surface stored in memory may then be read by a second display controller or display processing core for display on the second, e.g. external, display device. This will again reduce the amount of processing and memory bandwidth required for providing the output image on the second display device.
This arrangement can also reduce the processing required where a frame to be displayed is sent to an external display via wireless transmission. In this case, again, the composited output frame can be written back to memory and read therefrom for processing, e.g. by a video encoder, for wireless transmission, rather than, e.g., having to wirelessly transmit each respective input surface to a display controller for the external display for those surfaces then to be processed and composited by that display controller.
The Applicants believe that there remains scope for improvements to the operation of display controllers that include functionality to write output surfaces back to memory, such as display controllers of the form described in the Applicant's earlier UK Patent Application GB-A-2524359.
Like reference numerals are used for like components throughout the drawings, where appropriate.